SFNet: Learning Object-aware Semantic Correspondence

We address the problem of semantic correspondence, that is, establishing a dense flow field between images depicting different instances of the same object or scene category. We propose to use images annotated with binary foreground masks and subjected to synthetic geometric deformations to train a convolutional neural network (CNN) for this task. Using these masks as part of the supervisory signal offers a good compromise between semantic flow methods, where the amount of training data is limited by the cost of manually selecting point correspondences, and semantic alignment ones, where the regression of a single global geometric transformation between images may be sensitive to image-specific details such as background clutter. We propose a new CNN architecture, dubbed SFNet, which implements this idea. It leverages a new and differentiable version of the argmax function for end-to-end training, with a loss that combines mask and flow consistency with smoothness terms. Experimental results demonstrate the effectiveness of our approach, which significantly outperforms the state of the art on standard benchmarks.Comment: cvpr 2019 oral pape

Learning Semantic Correspondence Exploiting an Object-level Prior

We address the problem of semantic correspondence, that is, establishing a dense flow field between images depicting different instances of the same object or scene category. We propose to use images annotated with binary foreground masks and subjected to synthetic geometric deformations to train a convolutional neural network (CNN) for this task. Using these masks as part of the supervisory signal provides an object-level prior for the semantic correspondence task and offers a good compromise between semantic flow methods, where the amount of training data is limited by the cost of manually selecting point correspondences, and semantic alignment ones, where the regression of a single global geometric transformation between images may be sensitive to image-specific details such as background clutter. We propose a new CNN architecture, dubbed SFNet, which implements this idea. It leverages a new and differentiable version of the argmax function for end-to-end training, with a loss that combines mask and flow consistency with smoothness terms. Experimental results demonstrate the effectiveness of our approach, which significantly outperforms the state of the art on standard benchmarks

Learning Semantic Correspondence Exploiting an Object-level Prior

International audienceWe address the problem of semantic correspondence, that is, establishing a dense flow field between images depicting different instances of the same object or scene category. We propose to use images annotated with binary foreground masks and subjected to synthetic geometric deformations to train a convolutional neural network (CNN) for this task. Using these masks as part of the supervisory signal provides an object-level prior for the semantic correspondence task and offers a good compromise between semantic flow methods, where the amount of training data is limited by the cost of manually selecting point correspondences, and semantic alignment ones, where the regression of a single global geometric transformation between images may be sensitive to image-specific details such as background clutter. We propose a new CNN architecture, dubbed SFNet, which implements this idea. It leverages a new and differentiable version of the argmax function for end-to-end training, with a loss that combines mask and flow consistency with smoothness terms. Experimental results demonstrate the effectiveness of our approach, which significantly outperforms the state of the art on standard benchmarks

SFNet: Learning Object-aware Semantic Correspondence

International audienceWe address the problem of semantic correspondence, that is, establishing a dense flow field between images depicting different instances of the same object or scene category. We propose to use images annotated with binary foreground masks and subjected to synthetic geometric deformations to train a convolutional neural network (CNN) for this task. Using these masks as part of the supervisory signal offers a good compromise between semantic flow methods, where the amount of training data is limited by the cost of manually selecting point correspondences, and semantic alignment ones, where the regression of a single global geometric transformation between images may be sensitive to image-specific details such as background clutter. We propose a new CNN architecture, dubbed SFNet, which implements this idea. It leverages a new and differentiable version of the argmax function for end-to-end training, with a loss that combines mask and flow consistency with smoothness terms. Experimental results demonstrate the effectiveness of our approach, which significantly outperforms the state of the art on standard benchmarks

A Multi-Mode ULP Receiver Based on an Injection-Locked Oscillator for IoT Applications

This paper presents an ultra-low-power receiver based on the injection-locked oscillator (ILO), which is compatible with multiple modulation schemes such as on-off keying (OOK), binary frequency-shift keying (BFSK), and differential binary phase-shift keying (DBPSK). The receiver has been fabricated in 0.18-μm CMOS technology and operates in the ISM band of 2.4 GHz. A simple envelope detection can be used even for the demodulation of BFSK and DBPSK signals due to the conversion capability of the ILO from the frequency and phase to the amplitude. In the proposed receiver, a Q-enhanced single-ended-to-differential amplifier is employed to provide high-gain amplification as well as narrow band-pass filtering, which improves the sensitivity and selectivity of the receiver. In addition, a gain-control loop is formed in the receiver to maintain constant lock range and hence frequency-to-amplitude conversion ratio for the varying power of the BFSK-modulated receiver input signal. The receiver achieves the sensitivity of -87, -85, and -82 dBm for the OOK, BFSK, and DBPSK signals respectively at the data rate of 50 kb/s and the BER lower than 0.1% while consuming the power of 324 μW in total.1

A high efficiency low noise rf-to-dc converter employing gm-boosting envelope detector and offset canceled latch comparator

This work presents a high efficiency RF-to-DC conversion circuit composed of an LC-CL balun-based Gm-boosting envelope detector, a low noise baseband amplifier, and an offset canceled latch comparator. It was designed to have high sensitivity with low power consumption for wakeup receiver (WuRx) applications. The proposed envelope detector is based on a fully integrated inductively degenerated common-source amplifier with a series gate inductor. The LC-CL balun circuit is merged with the core of the envelope detector by sharing the on-chip gate and source inductors. The proposed technique doubles the transconductance of the input transistor of the envelope detector without any extra power consumption because the gate and source voltage on the input transistor operates in a differential mode. This results in a higher RF-to-DC conversion gain. In order to improve the sensitivity of the wake-up radio, the DC offset of the latch comparator circuit is canceled by controlling the body bias voltage of a pair of differential input transistors through a binary-weighted current source cell. In addition, the hysteresis characteristic is implemented in order to avoid unstable operation by the large noise at the compared signal. The hysteresis window is programmable by changing the channel width of the latch transistor. The low noise baseband amplifier amplifies the output signal of the envelope detector and transfers it into the comparator circuit with low noise. For the 2.4 GHz WuRx, the proposed envelope detector with no external matching components shows the simulated conversion gain of about 16.79 V/V when the input power is around the sensitivity of −60 dBm, and this is 1.7 times higher than that of the conventional envelope detector with the same current and load. The proposed RF-to-DC conversion circuit (WuRx) achieves a sensitivity of about −65.4 dBm based on 45% to 55% duty, dissipating a power of 22 µW from a 1.2 V supply voltage. © 2021 by the authors. Licensee MDPI, Basel, Switzerland.1

ReplaceNet: real-time replacement of a biological neural circuit with a hardware-assisted spiking neural network

Recent developments in artificial neural networks and their learning algorithms have enabled new research directions in computer vision, language modeling, and neuroscience. Among various neural network algorithms, spiking neural networks (SNNs) are well-suited for understanding the behavior of biological neural circuits. In this work, we propose to guide the training of a sparse SNN in order to replace a sub-region of a cultured hippocampal network with limited hardware resources. To verify our approach with a realistic experimental setup, we record spikes of cultured hippocampal neurons with a microelectrode array (in vitro). The main focus of this work is to dynamically cut unimportant synapses during SNN training on the fly so that the model can be realized on resource-constrained hardware, e.g., implantable devices. To do so, we adopt a simple STDP learning rule to easily select important synapses that impact the quality of spike timing learning. By combining the STDP rule with online supervised learning, we can precisely predict the spike pattern of the cultured network in real-time. The reduction in the model complexity, i.e., the reduced number of connections, significantly reduces the required hardware resources, which is crucial in developing an implantable chip for the treatment of neurological disorders. In addition to the new learning algorithm, we prototype a sparse SNN hardware on a small FPGA with pipelined execution and parallel computing to verify the possibility of real-time replacement. As a result, we can replace a sub-region of the biological neural circuit within 22 μs using 2.5 × fewer hardware resources, i.e., by allowing 80% sparsity in the SNN model, compared to the fully-connected SNN model. With energy-efficient algorithms and hardware, this work presents an essential step toward real-time neuroprosthetic computation

Adaptive input-to-neuron interlink development in training of spike-based liquid state machines

In this paper, we present a novel approach in developing input-to-neuron interlinks to achieve better accuracy in spike-based liquid state machines. An energy-efficient Spiking Neural Network suffer from lower accuracy in image classification compared to deep learning models. The previous LSM models randomly connect input neurons to excitatory neurons in a liquid. This limits the expressive power of a liquid model as large portion of excitatory neurons become inactive which never fire. To overcome this limitation, we propose an adaptive interlink development method which achieves 3.2% higher classification accuracy than the static LSM model of 3,200 neurons. Also, our hardware implementation on FPGA improves performance by 3.16∼4.99× or 1.47∼3.95× over CPU/GPU. © 2021 IEE